Performance Engineering Evaluation of Object-Oriented Systems with SPE*ED

Although object-oriented methods have been shown to help construct software systems that are easy to understand and modify, have a high potential for reuse, and are relatively quick and easy to implement, concern over performance of object-oriented systems represents a significant barrier to its adoption. Our experience has shown that it is possible to design objectoriented systems that have adequate performance and exhibit the other qualities, such as reusability, maintainability, and modifiability, that have made OOD so successful. However, doing this requires careful attention to performance goals throughout the life cycle. This paper describes the use of SPE•ED, a performance modeling tool that supports the SPE process, for early life cycle performance evaluation of object-oriented systems. The use of SPE•ED for performance engineering of object-oriented software is illustrated with a simple example. 1.0 Introduction Object-oriented development (OOD) methods have been shown to be valuable in constructing software systems that are easy to understand and modify, have a high potential for reuse, and are relatively quick and easy to implement. Despite the demonstrated successes of OOD, many organizations have been reluctant to adopt object-oriented techniques, largely due to concerns over performance. Our experience has shown that it is possible to design object-oriented systems that have adequate performance and exhibit the other qualities, such as reusability, maintainability, and modifiability, that have made OOD so successful [Smith and Williams, 1993]. However, doing this requires careful attention to performance goals throughout the life cycle. Failure to build-in performance from the beginning can result in the need to “tune” code, destroying the benefits obtained from a careful object-oriented design. In addition, it is unlikely that “tuned” code will ever equal the performance of code that has been engineered for performance. In the worst case, it will be impossible to meet performance goals by tuning, necessitating a complete redesign or even cancellation of the project. Software Performance Engineering (SPE) for object-oriented systems is especially difficult since functionality is decentralized. Performing a given function is likely to require collaboration among many different objects from several classes. These interactions can be numerous and complex and are often obscured by polymorphism and inheritance, making them difficult to trace. Distributing objects over a network can compound the problem. One of the principal barriers to the effective use of SPE with OOD is the gap between the designers who need feedback on the performance implications of design decisions and the performance specialists who have the skill to conduct comprehensive performance engineering studies with typical modeling tools. This gap means that extra time and effort is required to coordinate design formulation and analysis, effectively limiting the ability of designers to explore design alternatives. The ideal long-term solution to providing SPE assessments during the design stage is an evolution of today’s CASE tools to provide decision support for many facets of the design including correctness, completeness, performance, reliability, and so on. This approach, however, is not currently practical. It is too expensive for each CASE vendor to create their own modeling/analysis component. Therefore, we seek a nearterm capability to interface CASE tools to existing modeling tools. A previous paper defined the SPE information that CASE tools must collect [Williams and Smith, 1995]. This paper illustrates the translation from Object-oriented design models into performance models, and the use of the tool, SPE•EDTM,1 for early life cycle performance evaluation of object-oriented systems. SPE•ED is a performance modeling tool that supports the SPE process described in [Smith, 1990]. SPE•EDs software processing focus and automatic model generation make it easy to evaluate OOD architecture and design alternatives. Other features, such as the SPE project database and presentation and reporting features, support aspects of the SPE process other than modeling. The paper begins by reviewing related work. This is followed by an overview of SPE•ED. We then present an overview of the process of software performance engineering for object-oriented systems. A simple example illustrates the process. 2.0 Related Work Object-oriented methods typically defer consideration of performance issues until detailed design or implementation (see e.g., [Rumbaugh, et al., 1991], [Booch, 1994]). Even then, the approach tends to be very general. There is no attempt to integrate performance engineering into the development process. Some work specifically targeted at object-oriented systems has emerged from the performance community. Smith and Williams [Smith and Williams, 1993] describe performance engineering of an object-oriented design for a real-time system. However, this approach applies general SPE techniques and only addresses the specific problems of object-oriented systems in an ad hoc way. 1 SPE•EDTM is a trademark of Performance Engineering Services. Hrischuk et. al. [Hrischuk, et al., 1995] describe an approach based on constructing an early prototype which is then executed to produce angio traces . These angio traces are then used to construct workthreads (also known as timethreads or use case maps [Buhr and Casselman, 1992],[Buhr and Casselman, 1994], [Buhr and Casselman, 1996]), which are analogous to execution graphs. Workthreads provide empirical information about traversal frequencies for data-dependent choices and loops. Service times are estimated. This differs from the approach described here in that scenarios are derived from prototype execution rather than from the design and the system execution model is then generated automatically from the angio traces. Baldassari et.al. propose an integrated object-oriented CASE tool for software design that includes a simulation capability for performance assessment [Baldassari, et al., 1989, Baldassari and Bruno, 1988]. The CASE tool uses petri nets for the design description language rather than the general methods described above, thus the design specification and the performance model are equivalent and no translation is necessary. Using these capabilities requires developers to use both the PROTOB method and CASE tool. This paper uses the SPE tool SPE•ED to conduct the performance analysis. Other software modeling tools are available, such as [Beilner, et al., 1988, Beilner, et al., 1995, Goettge, 1990, Grummitt, 1991, Rolia, 1992, Turner, et al., 1992]. The approach described here could be adapted to other tools. Adaptation is necessary for these other tools that do not use execution graphs as their model paradigm. 3.0 SPE•ED Overview This section gives a brief overview of the features of the SPE tool that make it appropriate for OOD (and other) evaluations throughout their development life cycle. 3.1 Focus SPE•ED 's focus is the software performance model. Users create graphical models of envisioned software processing and provide performance specifications. Queueing network models are automatically generated from the software model specifications. A combination of analytic and simulation model solutions identify potential performance problems and software processing steps that may cause the problems. SPE•ED facilitates the creation of (deliberately) simple models of software processing with the goal of using the simplest possible model that identifies problems with the software architecture, design, or implementation plans. Simple models are desired because in the early life cycle phase in which they are created: • developers seldom have exact data that justifies a more sophisticated model, • they need quick feedback to influence development decisions, • they need to comprehend the model results, especially the correlation of the software decisions to the computer resource impacts. 3.2 Model description Users create the model with a graphical user interface streamlined to quickly define the software processing steps. The user's view of the model is a scenario, an execution graph of the software processing steps [Smith, 1990]. Software scenarios are assigned to the facilities that execute the processing steps. Models of distributed processing systems may have many scenarios and many facilities. Users specify software resource requirements for each processing step. Software resources may be the number of messages transmitted, the number of SQL queries, the number of SQL updates, etc. depending on the type of system to be studied and the key performance drivers for that system. A performance specialist provides overhead specifications that specify an estimate of the computer resource requirements for each software resource request. These are specified once and re-used for all software analysis that executes in that environment. This step is described in more detail later. 3.3 Model solution SPE•ED produces analytic results for the software models, and an approximate, analytic MVA solution of the generated queueing network model. A simulation solution is used for generated queueing network models with multiple software scenarios executing on one or more computer system facilities.2 Thus SPE•ED supports hybrid solutions the user selects the type of solution appropriate for the development life cycle stage and thus the precision of the data that feeds the model. There is no need for a detailed, lengthy simulation when only rough guesses of resource requirements are specified. 3.4 Model results The results reported by SPE•ED are the end-to-end response time, the elapsed time for each processing step, the device utilization, and the amount of time spent at each computer device for each processing step. This identifies both the potential computer device bottlenecks, and the portions of the device usage by processing step (thus the potential software processing bottlenecks). Model results are presented both with numeric values and color coding that uses cool colors to represent relatively low values and hot colors (yellow and red) calling attention to relatively high values. Up to 4 sets of results may be viewed together on a screen. This lets users view any combination of performance metrics for chosen levels in the software model hierarchy, and even compare performance metrics for design or implementation choices. An export feature lets users copy model results and paste them into word processing documents and presentation packages, or write out results for charting packages to create custom charts for reports. 3.5 Application areas SPE•ED is intended to model software systems under development. It may be any type of software: operating systems, database management systems, or custom 2 SPE•ED uses the CSIM modeling engine to solve the models [Schwetman, 1994]. applications. The software may execute on any hardware/software platform combination. The software may execute on a uniprocessor or in a distributed or client/server environment. 4.0 SPE Process Steps for OOD The process for performing SPE for an object-oriented design begins with a set of scenarios. A scenario is a description of the interactions between the system and its environment or between the internal objects involved in a particular use of the system under development. The scenario shows the objects that participate and the messages that flow between them. A message may represent either an event or invocation of one of the receiving object’s operations. The use of scenarios has become popular in many current approaches to objectoriented development. Scenarios, known as “use cases,” are an important component of Jacobson’s Objectory Method [Jacobson, et al., 1992]. Scenarios are also used in OMT [Rumbaugh, et al., 1991], Booch [Booch, 1994], Fusion [Coleman, et al., 1994], and the new Unified Modeling Language [Booch and Rumbaugh, 1995]. In objectoriented methods, scenarios are used to: • describe the externally visible behavior of the system, • involve users in the requirements analysis process, • support prototyping, • help validate the requirements specification, • understand interactions between objects, and • support requirements-based testing. Once the major functional scenarios have been identified, those that are important from a performance perspective are selected for performance modeling. Scenarios that are important to performance can be identified by a variety of techniques, including experience with similar systems and performance walkthroughs [Smith, 1990]. The scenarios are then translated to execution graphs (see below) which serve as input to SPE•ED. Currently, this translation is manual. However, the close correspondence between the way scenarios are expressed in object-oriented methods and execution graphs suggests that an automated translation should be possible. The next SPE steps are conducted after the translated model is entered into SPE•ED. Performance engineers enter data for the processing steps in the execution graphs, ensure that correct overhead specifications are in the SPE database, and evaluate model solutions for alternatives. These steps are illustrated with the following example. 5.0 Example To illustrate the use of SPE•ED for modeling and evaluating the performance of object-oriented systems, we present an example based on a simple automated teller machine (ATM). The ATM accepts a bank card and requests a personal identification number (PIN) for user authentication. Customers can perform any of three transactions at the ATM: deposit cash to an account, withdraw cash from an account, or request the available balance in an account. A customer may perform several transactions during a single ATM session. The ATM communicates with a computer at the host bank which verifies the account and processes the transaction. When the customer is finished using the ATM, a receipt is printed for all transactions and the customer’s card is returned. Here, we focus on scenarios that describe the use of the ATM. A full specification would include additional models, such as a class diagram and behavior descriptions for each class. However, our interest here is primarily in the use of scenarios as a bridge between Object-Oriented Development and Software Performance Engineering. Thus, these additional models are omitted. 5.1 Example Scenarios As described in [Williams and Smith, 1995], scenarios represent a common point of departure between object-oriented requirements or design models and SPE models. Scenarios may be represented in a variety of ways [Williams, 1994]. Here, we use Message Sequence Charts (MSCs) to describe scenarios in object-oriented models. The MSC notation is specified in ITU standard Z.120 [ITU, 1996]. Several other notations used to represent scenarios are based on MSCs (examples include: Event Flow Diagrams [Rumbaugh, et al., 1991]; Interaction Diagrams [Jacobson, et al., 1992], [Booch, 1994]; and Message Trace Diagrams [Booch and Rumbaugh, 1995]). However, none of these incorporates all of the features of MSCs needed to establish the correspondence between object-oriented scenarios and SPE scenarios. Figure 1 illustrates a high-level MSC for the ATM example. Each object that participates in the scenario is represented by a vertical line or axis. The axis is labeled with the object name (e.g., anATM). The vertical axis represents relative time which increases from top to bottom; an axis does not include an absolute time scale. Interactions between objects (events or operation invocations) are represented by horizontal arrows. Figure 1 describes a general scenario for user interaction with the ATM. The rectangular areas labeled “loop” and “alt” are known as “inline expressions” and denote repetition and alternation. This Message Sequence Chart indicates that the user may repeatedly select a transaction which may be a deposit, a withdrawal, or a balance inquiry. The rounded rectangles are “MSC references” which refer to other MSCs. The use of MSC references allows horizontal expansion of Message Sequence aUser anATM hostBank